Discrete- time classical and quantum Markovian evolutions: 
Maximum entropy problems on path space(^ 

Michele PavonH] 

Dipartimento di Matematica Pura ed Applicata, 
Universitd di Padova, via Trieste 63, 35131 Padova, Ital^ 

^ Francesco Ticozzjf] 

O 

;h 
< 

^ Abstract 



Dipartimento di Ingegneria dell'Informazione, 
Universitd di Padova, via Grademgo 6/B, 35131 Padova, Ital^ 

(Dated: April 29, 2009) 



The theory of Schrddinger bridges for diffusion processes is extended to classical and quantum 

Oh 

^ discrete-time Markovian evolutions. The solution of the path space maximum entropy problems 

> 

^ is obtained from the a priori model in both cases via a suitable multiplicative functional transfor- 

I— I mation. In the quantum case, nonequilibrium time reversal of quantum channels is discussed and 

space-time harmonic processes are introduced. 

cn 

^ PACS numbers: 05.30.-d,03.67.-a,03.67.Pp,02.50.-r 

Keywords: Markov chain, quantum channel, maximum entropy problem, Schrodinger bridge, time reversal 

' J evolution, space-time harmonic function. 

00 

o 

> 

X 



*Work partially supported by the MIUR-PRIN Italian grant "Identification and Control of Industrial Sys- 
tems" , by the GNAMPA-INDAM grant "Teoria del ControUo per Sistemi Quantistici" and by the Depart- 
ment of Information Engineering research project "QUINTET" . 



^URL: http : //www. math. unipd. it/'^pavon 



^Electronic address: pavon@math.unipd.it 



^URL: http : //www . dei .unipd. it/'^ticozzi 

^Electronic address: 



ticozziOdei.unipd.it 



1 



I. INTRODUCTION 



In this paper, we study certain maximum entropy problems for discrete time and discrete 
state space Markov evolutions first considered by Erwin Schrodinger in the early thirties 
for diffusion processes [l9l 150] . In these problems, there is an a priori distribution on path 
space. Then new information becomes available in the form of the initial or terminal (or 
both) marginal distribution. One seeks a new path space distribution that has the correct 
marginal(s) and minimizes relative entropy from the prior distribution. Given the diffusion 
case results, their extension to Markov chains turns out to be rather straightforward. These 
results serve us as a guideline and for comparison purposes in the more challenging quantum 
case. 

To the best of our knowledge, the only previous discrete-time paper on this topic is [1], 
which deals with the continuous state space case. The key results on Schrodinger bridges, 
however, are there merely stated and the most delicate points in this extension, such as 
positivity of the space time harmonic function needed for the solution and existence and 
uniqueness of the latter, are altogether ignored. Here we show that the solution process 
can be obtained, in analogy to the diffusion case, via a suitable multiplicative functional 



transformation of the "prior" Markov process, see Theorem IV. 1 As in the diffusion case. 



an abstract result of Beurling and Jamison (Theorem IV. 2) can be used to prove existence 
and uniqueness of the solution of the Schrodinger system for finite, irreducible and aperiodic 
Markov chains, see Corollary IV. 4[ 

In order to derive corresponding results for quantum channels, we first need to develop 
various kinematical results. These concern extending the results on time-reversal of the chan- 
nel by [3], [H], and developing space-time harmonic processes. We also need to introduce a 
suitable concept of quantum "trajectory": We consider a sequence of orthogonal projec- 
tions selected from the spectral representations of a time-ordered sequence of observables. 
In spite of the obvious difficulties one can expect from the non commutative structure, we 
are actually able to solve two key maximum entropy problems on path space, cf. Theorems 
VI. 4 , 1 VI. 5 In the second case, the solution does not depend on the particular "quantum 



path" chosen. Moreover, with the appropriate understanding of objects and properties, in 
both cases it bears a striking similarity to the classical case. 

The outline of the paper is as follows. In Section II, we outline the theory of Schrodinger 
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bridges for diffusion processes. In Section III, as a warm up, we solve the simpler problems 
where only the initial or final marginal is prescribed. These are later needed for the pur- 
pose of comparison in the quantum case. The problem where both marginals are fixed is 
solved in Section IV. Section V is devoted to introduce a class of Markovian processes from 
statistical quantum mechanics. In particular, their time reversal and space-time harmonic 
evolutions are there considered. In Section VI, after introducing a suitable concept of path, 
two maximum entropy problems are discussed and solved. The paper concludes with some 
final remarks and an outlook on potential developments in Section VII. 



II. CLASSICAL SCHRODINGER'S BRIDGES 



In 1931/32, Erwin Schrodinger studied the following problem |l9l [50]. Consider the 
evolution of a cloud of N independent Brownian particles. This cloud of particles has been 
observed having at some initial time to an empirical distribution equal to po{x)dx. At some 
later time ti, an empirical distribution equal to pi{x)dx is observed which considerably 
differs from what it should be according to the law of large numbers {N is large, say of the 
order of Avogadro's number), namely 



p{to,y,ti,x)po{y)dy dx, 



to 



where 



p{s,y,t,x) = [27r(t - s)] 2 exp 



\x-y 



21 



s <t 



2{t - s) 

is the transition density of the Wiener process. It is apparent that the particles have been 
transported in an unlikely way. But of the many unlikely ways in which this could have 
happened, which one is the most likely? 

Schrodinger considered first the simpler problem of the Brownian bridge, namely the 
case where the endpoint distributions are Dirac measures. He introduced the three times 
(two-sided) transition density 

p{s,x;t,y)p{t,y;u, z) 



q{s,x;t,y;u,z) 



s < t < u, 



p{s,x;u,z) 

which the "prior" and solution measures on path space must share. He then considered joint 
distributions of the initial and final time of the form 



p{B) 



p{to,x;ti,y)d{vo x vi){x,y). 



(1) 



B 



where B is a Borel set in x M", and z/q and z/i are cx-finite measures on M". This 
form of the endpoint joint distribution, as proven by Jamison in [27], characterized derived 
processes that have the Markov property. General endpoint measures lead to reciprocal 
processes (one-dimensional Markov fields) introduced by Bernstein in [6j. Using then a 
coarse graining approach on the initial and final state typical of Boltzmann and Lagrange 
multipliers, Schrodinger went ahead to compute the most likely endpoint distribution in the 
class ([T]) under the "prior" p{s,x,t,y) and with the prescribed marginals. It turned out 
that the solution, namely the bridge from po to pi over Brownian motion, has at each time 
a density q that factors as q{x,t) = ip{x,t)0{x,t), where ip and ip are a p-harmonic and a 
p-coharmonic functions, respectively. That is 

Lp{t, x) = j pit, X, ti, y)ip{ti, y)dy, (2) 

0{t, ^) = J P(^o, y, t, x)(p{to, y)dy. (3) 

The existence and uniqueness of a pair (p) satisfying (|2|-([3]) and the boundary conditions 
ip{x,tQ)ip{x^to) = po{x), ip{x,ti)ip(x,ti) = pi{x) was guessed by Schrodinger on the basis of 
his intuition. He was later shown to be quite right in various degrees of generality by Fortet 
[25] , Beurling [7], Jamison [27], FoUmer [22]. Jamison also showed [2B] that the solution 
is a Markov diffusion process with drift given by b{x,t) = Vlogy?, i.e. its generator is 
C = Vlog<y9 ■ V + ^A. It is namely an /i-path process in the sense of Doob [19]. Similar 
results may be obtained when the "reference" process is any finite-energy Markovian diffusion 
possibly with creation and killing, see e.g. [55] which is a survey paper with an extended 
bibliography. It took more than fifty years before FoUmer observed in [22j that all of this 
can be cast in the framework of a problem of large deviations of the empirical distribution 
on path space [13, [TBI ED] , see also Appendix [C} 

III. MAXIMUM ENTROPY PROBLEMS FOR MARKOV CHAINS 
A. Kinematics of Markov chains 

Consider a Markov chain X = {X{t); t = 0, 1, 2, . . .} taking values in the finite or count- 
ably infinite set X. Since X is countable, we can identify X with a subset of N. Let us 
introduce the distribution of X(t) given by pi{t) = P(X(t) = i) and the transition prohahil- 
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ities Pijit) := P(X(t + 1) = j\X(t) = i). They are connected through 

P,{t + l) = Y,P^A^)P^i^)■ (4) 

i 

Let us agree that f always indicates adjoint with respect to the natural inner product. 
Hence, in the case of matrices, it denotes transposition and, in the complex case below, 
transposition plus conjugation. We can then rewrite Q as p{t + 1) = P\t)p{t), where 
pit)"^ = {po(t),pi(t),p2(t), . . .) and P(t) = {pij(t)) is the transition matrix. The latter is 
stochastic, i.e. all elements are nonnegative and rows sum to one. Let us introduce the 
reverse-time transition probabilities 

q,i{t,p{0)) := F{X{t) = z\X{t + l)=j), (5) 

where we have emphasized the dependence on the initial distribution p{0). Also notice that 
the qji depend on time even when the pij don't. Let us find the relation between the qji and 
the Pij. 

p,(t)p,,(t) = P(X(t) = t,Xit + 1) = j) = p,it + l)g,,(t,p(0)). (6) 

Notice that for pj{t + 1) = 0, qji{t,p{0)),i G N may be defined arbitrarily any number 
between zero and one provided 

^g,,(t,p(0)) = L 

i 

Whenever pj(t + 1) > 0, we get the relation 

QAt,Pm = ^^^^P^At)■ (7) 



Definition III.l A function h : Nx X ^ W is called space-time harmonic for the transition 
mechanism {P{t); t = 0,1, . . .} of a chain if, for every t > and all i,j e X, it satisfies the 
backward equation 

h{t,z) = J2PvitMt+^,j)- (8) 

j 

Space-time harmonic functions, a terminology due to Doob and motivated by the case of 
diffusion processes, play a central role in constructing Schrodinger bridges. They are closely 
related to a class of martingales that are instantaneous functions of X{t), see the Appendix 
|X]for definition and properties. 
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B. Simple path-space maximum entropy problems 



We consider first the simpler maximum entropy problems where only the initial or final 
marginals are prescribed. Although they turn out to be almost special cases of the initial- 
final marginal problem, they are important for statistical mechanics. 

Definition III. 2 Letp and q be probability distributions on a finite or countably infinite set. 
We say that the support of p is contained in the support of q if qi = pi = and write 
supp{p) C supp{q). The Information Divergence or Relative Entropy or KuUback-Leibler 
Index of q from p is defined to be 

^^^11^^ ^ I E.P(01ogf|, suppip) C supp{q), ^ 
I +00, supp{p) ^ supp{q). 

where, by definition, ■ logO = 0. 

Let X = {X(0),X(1), . . .} be a Markov chain with state space X, transition probabilities 
(7r.y(t)) and marginal probabilities F{X{t) = i) = TTi{t). Let 11 denote the corresponding 
joint distribution of {X{0),X{1), . . . ,X{T)} (distributions on X'^'^^ are always denoted by 
capital, boldface letters). Let V{0,T;p^) denote the family of Markovian distributions P 
on X'^'^^ that have marginal p^ at time T, respectively, and have support contained in the 
support of n. Consider the 
Maximum Entropy Problem (MEPl): 

minimize {B{P\\U);P e V{0,T;p^)} . (10) 

Observe that the constraint on the final marginal in Problem (MEPl), may be expressed as 
a linear constraint on P as 

^ ^ • • • ^ P(io, «i, • • • , ir-u ir) = P^(^t)- 

Thus Problem (MEPl) has the standard form of a maximum entropy problem |13]. Observe 
moreover that existence of a minimum point is always guaranteed since we are minimizing 
the lower semicontinuous function D(-||n) with compact level sets [20J over the closed set 
T>{0,T;p^). The same applies to the other maximum entropy problems considered below. 



In order to facilitate the solution of Problem (MEPl) in (10), let us introduce the reverse 



time transition probabihties qji{t) (Isl) corresponding to P and qlit) corresponding to 11. By 



the Markov property, we have 

n(^o, zi, . . . , tT-i, zt) = gr,.o(o) ■ ■ C.T-.(^ - 1) ■ ^^t(^)- (12) 

We then get the following representation for D(P||n). 

Lemma III. 3 In the previous notation, assume that the support of P is contained in the 
support o/n. Let p(t) and 7i{t) denote the marginals of P and II at time t, respectively. 
Then 

T 

D(P||n) = Y,Y.^(q,,,,_,{k - l)\\ql,,_,{k - 1)) + D(pi7r(r)). (13) 

k=l ik 

The elementary proof is deferred to the Appendix | 



Theorem III. 4 A solution to (MEPl) (10) is given by the distribution P corresponding 
to the Markov chain with marginal distribution p^ at time T and reverse time transition 
mechanism equal to that of II, namely 

W.(^-l)=C..-.(^-l)' k = l,2,...,T. (14) 



Proof. Since both terms in (13) are nonnegative, and D(j9^ ||7r(T)) is invariant over V{p^), 



the best we can hope for, when minimizing D(P||n), is to make the first term equal to zero. 



This is the case if (14) holds true. □ 



Remark III. 5 A moment thought reveals that the argument can be readily extended to show 
that P is optimal in the larger class of all distributions on X'^^^ that have marginal p^ at 



time T. Indeed, it suffices in Lemma III. 3 to replace forward and reverse-time transition 
probabilities with conditional probabilities given the whole past or whole future trajectory, 
respectively. The Markovian nature of the solution P therefore emerges as a by-product 
result of the variational problem. The same observation applies to the maximum entropy 
problems (MEP2) and (MEP3) below. 

Let us compute the forward transition probabilities of P. Let p{t) and Pij denote the 
marginal at time t and the forward transition probabilities of P, respectively. Let vTjj be the 
forward transition probabilities of 11. By (|6|), we have 



Pi{t)p^,{t) = p,{t + l)g,,(t), 7r,(t)7r,, = Tx,{t + l)gj,(t). (15) 
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Assume now that 7ii{t) > 0,pi{t) > 0,\/i,0 < t < T. Then (pi) and iK^ yield 



(16) 



Observe, moreover, that if is space-time harmonic with respect to the transition mechanism 



of n since, by (16), 



3 j 

Hence, the solution is obtained from the reference probability 11 through a "multiplicative 
functional" transformation in analogy to the diffusion case [28j. Consider now the case where 
the initial marginal is fixed. Let Pij{k) denote the transition probabilities of P. Using 

P(X(0) = 2o,X(l) =Zi,...,X(n) =z„) =4-p,„,,(0)---p,„_,,„_,(n-2)-p,_,,Jn-l), (17) 



and the corresponding one for 11, we get along the lines of Lemma III. 3 the representation 
of D(P||n): 



T-l 



fPlini 



Kp>m + Y,J2^iP^.^Um^^.^..M)P^M■ 
k=0 ik 



(18) 



We then get the following result. 



Theorem III. 6 A solution to the problem of maximizing {D(P||n); P G I?(0, T; p°)} 
(MEP2) is given by the distribution P corresponding to the Markov chain with marginal 
distribution p^ at time and forward transition mechanism equal to that of II, namely 



Pijit) = nij{t), t = 0, 1,...,T. 



(19) 



Let us compute the reverse-time transition probabilities of P. Let p{t) and qji{t) denote the 
marginal at time t and the reverse-time transition probabilities of P, respectively. Let qji{t) 
be the reverse-time transition probabilities of Tl. By ([6]), we have 



Pj{t + l)qji{t) = Pi{t)7iij{t), Tii{t)'Kij = 7ij{t + l)q]i{t). 



(20) 



Assume now that TTi{t) > 0,Pi{t) > 0, Vz, < t < T. Then ([20]) and ([19]) yield 

9{t,z) 



e{t + i,j 



e{t,z):- 



mit)' 



(21) 
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Observe, moreover, that 9 is space-time harmonic with respect to the reverse-time transition 



mechanism of 11 since, by (21 ) 



Again the solution is obtained from the reference probabihty 11 through a "multiphcative 



functional" transformation. The special case of property (22) where vTy do not depend on 
time and p{t) = vr a stationary distribution is strictly connected to a strong form of the 
second law and a principle of minimum dissipation [51]. We finally mention that Problems 
(MEP1)-(MEP2) are connected to optimal stochastic control problems [45j (corresponding 
results for continuous time Markov chains are contained in an unpublished 1988 Master 
Thesis by C. M. Pelaggi [17]). 

IV. SCHRODINGER BRIDGES FOR MARKOV CHAINS 
A. The initial-final marginal problem 

In the same notation as in the previous section, let P(0,T;p°,p^) denote the family of 
Markovian distributions P on X^^^ that have marginals at time and p^ at time T, 
respectively, and have support contained in the support of 11. We consider the following 
Maximum Entropy Problem (MEP3): 

minimize {D(P||n); P G r'(0,T;p°,p^)} . (23) 



Consider the representation (18). Since D(p ||7r(0)) is invariant over T'(0,T;p ,p ), Problem 



(MEP3) in ([23|) is equivalent to 

minimize \Y.Y.^{Pi,^,^Am^^,^.Uk))P^k{k)■,'P E P(0,T;/,/)| . (24) 

U=0 ik ) 

Rather than following the standard Schrodinger-FoUmer approach as in [4i|, we develop a 
discrete counterpart of the approach of Nagasawa for diffusion processes [IQ] which relies 
heavily on Girsanov's theory [29]. Consider a space-time harmonic function for the refer- 
ence stochastic evolution 11, namely 

^{t,i) = Y,'^,,{t)^{t + l,3), < t < (T- 1). (25) 



Assume that ^{t,i) > 0, VO < t < T, Vi G Af. Then Problem (24) is equivalent to 

minimize {J(P);P G 15(0, r;p°,p^)} , (26) 



where 



T-l 



A:=0 Ik 



iT ' 



since the last two terms are invariant over D(0,T;p ,p ). Next, observe that 



.0 ^l) 



T-l 



Moreover, 

= E <^(fc, (A;) - E log + 1' ^k+i) E (^) 

ik ik+1 ik 

= EE^*fe^fc+i*^^) [log(^(A;,4) -log(^(/c + l,ifc+i)]pi,(/c). 



Hence, J(P) may be rewritten as 

T-l 



fc=0 ik 



V9(A; + l,ifc+i) 



Define 



Pij{k) = 7iij{k) 



Notice that pij > and, by (p5|). 



9?(A;,z) 

Consider the probabilities p{t) defined by the recursion 



Define 



ip{t,i)' 



(27) 
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We get 

(28) 

namely 0{t,i) is space-time co-harmonic. We have therefore estabhshed the following theo- 
rem: 

Theorem IV. 1 Suppose there exists a pair of nonnegative functions {^p,0) defined on 
[0, T] X X and satisfying the system 

ip{t, t) = J2 ^^A^Mi + 1' j), < t < (T - 1) (29) 

j 

(p{t + 1, j) = 0, < t < (T - 1), (30) 

i 

as well as the boundary conditions 

¥p(0,0-<^(0,0 :=p°, i^{T,i)-^{T,i):=p\, W G A". (31) 

Suppose moreover that (p{t, i) > 0, VO < t < T, Vz G X . Then, the Markov distribution P in 
V{0,T;p^,p^) having transition probabilities 



(p[t,t) 



solves problem (MEP3) (|23 



Notice that if (v?, <^) satisfy (29)-(30)-(31 ), so does the pair (cy?, ^(p) for all c > 0. Hence, 
uniqueness for the Schrodinger system is always intended up to such multiplications. As 
in the diffusion case, the problem is now reduced to establish, under suitable assumptions. 



existence and uniqueness for the Schrodinger system (29)-(30)-(31) (notice that this issue 
is not even mentioned in |3]). Also observe that the solution of (MEPl) and (MEP2) may 
be viewed as a special cases of the solution of (MEP3) where the role of the space-time co- 
harmonic function ip(t,i) is played by the prior probabilities Hiit). Indeed, let us compute 
the reverse-time transition probabilities qji{t) of P. By ^ and using pi{t) = ip{t, i) ■ <^(t, i), 
we get 

^ u\ Pi(t) . ..x Pi{t) (p{t + l,j) 
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where 



Moreover, 

namely, C, is space-time harmonic with respect to the reverse-time transition mechanism of 
n. We then get the suggestive formula 

Pi{t) = ip{t,i) -^{t^i) ■ 7Ti{t), 

showing that the "a posterior" marginals are obtained from the "a priori" marginals through 
multiplication by a space-time harmonic function for the forward transition mechanism and 
by a space-time harmonic function for the reverse time transition mechanism. In problems 
(MEPl) and (MEP2), we have C,{t,i) = 1 and <f{t,i) = 1, respectively. Namely, the solu- 
tions to (MEPl) and (MEP2) appear as particular cases of the solution to (MEP3), where 
only the forward or only the backward transition mechanism undergoes a multiplicative 
functional transformation induced by a space-time harmonic function, whereas in (MEP3) 
both transition mechanisms are subject to such a transformation. Finally, notice that if we 
exchange p° and p^, the solution will simply be the time reversal of P: This intrinsic time 
reversibility of the bridge attracted the interest of Schrodinger and Kolmogorov, see the 
titles of [31119]. 



B. Existence and uniqueness for the Schrodinger system 



Existence and uniqueness of the solution to the Schrodinger system (29)-(30)-(31) follows 



from a very deep result of Beurling |7], suitably extended by Jamison [271 Theorem 3.2]. 

Theorem IV. 2 Let S be a a -compact metric space. Let /ii and /i2 be probability measures 
on its a- field S of Borel sets. Suppose p is a continuous, strictly positive function on S x S . 
Then there exists a unique pair fj,, vr of measures on S x S such that 

1. fi is a probability measure and ti is a a -finite measure; 

2. fi{E X S)= fii{E), i2{S xE) = i22{E), E e ^; 
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It is argued [27| pp. 76-77] that the existence and uniqueness of this theorem are equivalent 
to existence and uniqueness for an abstract Schrodinger system where fii and fi2 play the role 
of initial and final marginals, respectively, and p the role of the Markov transition density of 
the prior probability on path space. The fact that [27] deals with the continuous time case 
is immaterial: Only the initial and final times are involved in this result. 

In order to apply the Beurling-Jamison theorem to our setting, we first need a few 
observations. In our setting, S = X is finite or countably infinite. We endow X 
with the discrete metric, namely d{x,y) = 1 for all pairs {x,y) G X x X,x ^ y, and 
d{x,x) = 0,Vx G X. This makes X a cr-compact metric space where the transition prob- 
ability p{s,x,t,y) := F{X{t) = y\X{s) = x) is continuous on X x X for any pair of times 
s < t. The Borel a-field of X, denoted B{X) is then simply the family of all subsets of X. 
The probability density with respect to the counting measure Ax is simply the probability 
of that element in X. 

Theorem IV. 3 Let X = {X{0), X{1), . . .} be a Markov chain with state space X and 
transition probabilities TTij{t). Assume 

1. p^ is a distribution on X with p\ > 0, Va; G X ; 

2. p{0,x,T,y) > 0,^x,y G X. 



Then the Schrodinger system {29)-(30)-(31) has a unique solution with ip{t,x) > 0, VO < 
t<TyxeX. 



Proof. Existence and uniqueness for the Schrodinger system follows from Theorem |IV.2 
From (31), the nonnegativity of if and the assumption on p^, it follows that (p{T,x) > 



0,Wx G X. Recalling that 

5^7r,,(t) = l, Vt, 



we get from (29) that ip{t,x) > 0, VO < t < T, Vx G A'. □ 



In many important applications, the prior transition probabilities do not depend on time. 
We get the following result for finite, irreducible and aperiodic Markov chains. 

Corollary IV. 4 Let {X{0), X{1), . . .} be a Markov chain with finite state space X and 
transition matrix U = (vTy). Assume 
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1. is a distribution on X with p]. > 0, Vx G A"; 

2. the matrix has all positive elements. 



Then the Schrddinger system (29)-(30)-(31 ) has a unique solution with (p{t,x) > 0, VO < 
t<T,\/xe X. 

The implications of these results for large deviations is outlined in the Appendix [C] 

V. QUANTUM MARKOV CHANNELS 

In the following sections, we show how the framework developed for classical Markov 
chains can be extended to the class of discrete-time, Markovian quantum processes generated 
by families of quantum operations. We begin by establishing some notation and recalling 
some basic facts from quantum statistical theory. Some classical references are [25], HHJ |53] . 
A basic and clear exposition from the quantum information viewpoint can be found in [H] , 
while an excellent introduction to quantum probability concepts and formalism is [37]. 

A. Quantum probabilities, entropy and quantum operations 

Consider a finite-dimensional quantum system Q, with associated Hilbert space Hq iso- 
morphic to C". In the quantum probability formalism, random variables or observables 
for the system are represented by Hermitian matrices X G Sj{n). They admit a spectral 
representation X = Ylj^j^j^ where each real eigenvalue xj represents the random out- 
come associated to the quantum event corresponding to the orthogonal projection Ilj. The 
role of the probability distributions is played here by positive-definite, unit-trace matrices 
p > 0,trace(p) = 1, called density matrices. The set D{n) of density matrices is convex and 
has the rank-one orthogonal projections as extreme points. Assume that the density matrix 
associated to the state of the system is p. The probability of measuring xj, or in general the 
probability associated to the quantum event Uj, is given by: 

Fp{Uj) = trace (Hj-pHj). 
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If the outcome corresponding to an event Uj has been measured, the density matrix condi- 
tioned on the measurement record is: 

^1"^ = tracen,pn,"^^"^- 
Hence, the joint probabihty of obtaining 11^ after Uj in subsequent measurements can be 
computed by: 

Pp(n„ Hfc) = trace(nfcn,pn,nfc), (33) 

where the order of events is relevant. Similarly one obtains nested expressions for joint 
probabilities of arbitrary event sequences. Expectations are then computed by the trace 
functional: 

Ep(X) = Xjtrace(njpnj) = trace(pX). 

j 

Notice that this implies that if the measurement has occurred, but the outcome has not 
been recorded, the correct conditional density matrix is: 

which is in general different from the pre-measurement p, in contrast with the classical case. 
We refer to these "blind" measurement processes as non-selective measurements. 

For any matrix M, the support of M, denoted supp{M), is the orthogonal complement 
of ker(M). Given two density matrices p, a, the quantum relative entropy is defined by: 

J trace(p(logp-logcr)), supp{p) C supp{a), 

I +00, supp[p) n supp[o) fi 

As in the classical case, quantum relative entropy has the property of a pseudo-distance (see 
e.g. [m The Klein's Inequality D(p||cr) > holds, equality occurring if and only if 

p = a. Moreover, quantum relative entropy is continuous where it is not infinite and it is 
jointly convex, but not symmetric, in its arguments. 

A wide class of physically relevant, Markovian transition mechanisms are represented by 
linear. Trace Preserving and Completely Positive (TPCP) maps from density matrices to 
density matrices. A TPCP map S\ in turn, can be represented by a Kraus operator-sum 
i.e.: 

Pt+i=S\pt) = J2M,PtMl 
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where the nxn matrices Mj must satisfy MjMj = / in order for S"^ to be trace preserving. 
[57] Following common conventions in quantum information [44J, we refer to general Kraus 
operator-sums as Kraus maps, and to trace-non-increasing ones as quantum operations. 
As an example that will be used in the following Sections, notice that the non-selective 
measurement process = Ylj^jP^j ^ valid TPCP map, while the un-normalized 

conditioning £^^(p) = UjpUj is a quantum operation, since it is trace non-increasing. 

The action of the dynamics on observables can be derived by duality with respect to the 
Hilbert- Schmidt inner product: 

tTace{X£\pt)) = trace(£^(X)pi), 

where S{X) = J2j M]XMj. It follows that if jg trace -preserving, then £{■) is identity 
preserving and vice-versa. 

We are now in a position to introduce a quantum analogue of space-time harmonic func- 
tions. Consider a quantum Markov process, generated by po and a sequence of TPCP maps 

{^/}tG[0,T-l]- 

Definition V.l (Quantum space-time harmonic property) A sequence of observ- 
ables {Yt}te[a,T-i] is said to be space-time harmonic with respect to the family {St}te[o.T-i] 

Yt = St{Yt+i). (35) 

As in the classical case, space-time harmonic processes will be shown to play a central role 
in the solution of maximum entropy problems on path spaces. 

B. Time- reversal of quantum operations 

Another key ingredient in the study of maximum entropy problems on path space, is, very 
much like for classical Markov chains, the reverse-time transition mechanism. We give here 
an intuitive derivation of the time-reversal of a quantum operation through an analogy with 
the classical case, referring to [51] for a thorough discussion in the framework of adjoint 
Markov semigroups [Sj, its connections to the equilibrium case [H] and previous results 
from the quantum error correction literature |3], EH] ■ 

If we write pt = '^■piUt^i, pt+i = J2j Ij^t+^jy '^^^ think of the density matrices as 
classical probability densities over the abelian algebras of quantum events generated by their 

16 



spectral families. The probability of measuring Uj^t+i, after Ilj ^ has been measured at time 
t and S"^ has acted on the system, is given by: 



trace \Uj^t+iJ2^k^^'tPt^i't^l^i'^+^ 
trace ^H,- t+i ^ M^n.^^M^^n.-i+i j pi 



(36) 



obtaining an analogous of the transition probabilities in the classical case. Following this 
analogy, the reverse-time T^l^p^ is then required to satisfy: 

p^t,p,(n,,i,n,,i+i) = trace (uj^tJ^M^^ pt)^i,t+iRl{£, pt)T^j,^ Qj = P^t ^^,^^^^(n,,i+i, n,,o. 

(37) 

Notice that (36)-(37) provide us with the equivalent of (|6| for classical chains. Assume for 



now that pt+i is full-rank. From (36), using the cyclic property of the trace and the fact 



that pt+i and Hj^t+i commute for all j, we obtain: 

k 

= 2^ trace lUi^tp? Mlpt^\Uj^t+ipt+i'nj,t+iPt+i^kPt'^i,i 



trace [ H,, (p|Mtpi7i) n.-^+i (p,7iM,p|n,i) j q^. 



Hence, we define 



and a Kraus map: 



Rji£,Pt) = Pt+\M,p'} 



2 



Y,RA^^pt){-)Rp,Pt)- 



(38) 



(39) 



In [51] , we show that this map is in fact a quantum operation, that it can be augmented 
to a trace-preserving quantum operation, and that it is the correct time-reversal for S with 
respect to the initial density pt- This is established also in the case rank(pt_|_i) < n, thereby 
extending the results in [3] (in this case, the inverse has to be replaced by the Moore-Penrose 
pseudoinverse, cf. [26]). 

For any p and S"^ with Kraus operators {M^}, define the map Tp from quantum operations 
to quantum operations 

Tp-.S^ ^TpiS^), (40) 
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where Tp{£''') has Kraus operators {p^ Ml{£{p))~^}. The resuhs of [3] show that the action 
of Tp is independent of the particular Kraus representation of f"^. With this definition, we 
have that 



n 



Theorem V.2 (Quantum Operation Time Reversal, |51| ) Let E'^ he a TPCP map. 
U Pt+i = ^KPt)i then for any pt G D{n), TZ^ p^{-) defined as in (39) is the time-reversal of 
E for Pt, I.e.: 

Pt = Tp,{E^) = nl^Xpt+,), (41) 



and 



Tp,,,inlji^t) = E\at) 



(42) 



for all at G 2) (7^) such that supp{at) C supp{pt). Moreover, it can he augmented to he 



TPCP without affecting properties (41)-(42). 



Remark V.3 Notice that if pt is full rank, (42) implies that Tp^^^oTp^ is the the identity map 
on quantum operations, i.e. the time reversal of the time-reversal is the original forward map. 
One can also show that in general the time-reversal mechanism is not unique just as in 
the classical case (see the discussion following ^ in Section III A). While studying quantum 



error correction prohlems J^, the same p{-) has heen suggested hy Barnum and Knill as 
a near-optimal correction operator. In their setting, E'^{p) is full-rank, and represents the 
output of a channel E with input state p = "^jPjPj, a statistical mixture of some quantum 
codewords of interest to he recovered. It has also heen proven there that IZl pi^-) is independent 
of the particular Kraus representation of E. 

Given a quantum Markov process, generated by po and a sequence of TPCP maps 
{Et}te[o,T-i], a sequence of observables {Yt}t(z[o,T~i] is said to be space-time harmonic 
reverse-time with respect to the family {7l£^^p^}t<^io,T-i] if: 



zn 



Yt+i — '^et,pt(Xt), 



(43) 



extending Definition V.l in analogy with the classical case. 
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VI. MAXIMUM ENTROPY PROBLEMS FOR QUANTUM MARKOV CHAN- 
NELS 



In the quantum case, the definition of a path-space for a Markov process is not obvious. 
Here, we build up quantum trajectories associating at each time an observable quantity 
and conditioning the state and the evolution to measurements of such observables. We 
next proceed by formulating and solving quantum versions of problems MEP1-MEP2. The 
framework we develop leads to results that are in striking analogy with the classical case. 
The third case, where both marginals are prescribed, presents some intrinsic difficulties when 
one tries to generalize the classical approach, essentially due to the definition of quantum 
relative entropy. A discussion of this problem is deferred to future publications. This issue 
is strictly related to the kinematical structure we highlight in [51], where an alternative 
definition of quantum relative entropy emerges as a natural candidate. Further work is 
also needed to address the possibility of extending the framework to include generalized or 
indirect measurement on the system, as it is typically the case in quantum optics [T2^] . In our 
setting, anyway, commutativity of observables associated to different times is not required. 



A. Path spaces for quantum Markov processes 

Consider a quantum Markov process for a finite dimensional system Q with associated 
Hilbert space Hq, generated by an initial density matrix do and a sequence of TPCP maps 
{S}}teio,T~i], with each Sj admitting a Kraus representation with matrices {Mfc(t)}. 

We define a set of possible trajectories, or quantum paths, by considering a time-indexed 
family of observables {^t}, Xf = YlJ^i^i^ii^)^ with t G [0,T]. The paths are then all the 
possible time-ordered sequences of events (IliQ^O), nj^(l), . . . , Ili^(T)) , with it E [I, nit]. By 



using (33), we can compute the joint probability for a given path with the nested expression: 
4o,n,....T)(^o) = trace (n,,(T)4_,(n,,_,(T - 1) . . . £:t(n,,(0)aon,„(0)) . . .)n,^(r)) . (44) 
Lemma VI. 1 Define the path-conditioned density matrices for t G [0,T] via the relations 
a£,o = $^nio(0)aon,„(0), 

ae,t+i = S}{ae,t) = 5^n,,^,(t + l)4(a^,i)n,,^,(t + 1), (45) 



H+l 
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where Sj is TPCP and can be represented with double-indexed Kraus operators \Jii{t + 
l)Mfc(t)}. The marginal distribution w^((Jo) at time t G [0,T], is then given by: 



>o) = trace{ll,,{t)aeA,{t)) ■ (46) 



Proof. From (44), by using the cyclic property of trace and the fact that is trace 



preserving, we get: 



E4o.i,....t)(^o) = traceM 5^n,,(T) j4^i(n,,_,(T-l)...ft(n,„(0)aon,„(^^ 



^{io,n,...,^T-i)(^o)- 



Hence, by iterating and substituting the definition (45): 

io,...,it-i,it+l,---,iT io,---,it-i 

= trace (n,, {t)sl, ( n,,_, (t - 1) . . . ^ ( ^ (o)cron,„ (0)) . . . n,,_, (t - 1)) n,, (t) 

H-i io 

= tmce(lli^{t)a£^Jli^{t) 



□ 

Remark VI. 2 If for eacht [Xt,(yt] '■= XtO't — o'tXt = 0, then, computing the reduced density 
operators at each time, we obtain the original quantum Markov process, that is at = (Tt for all 
t. In general this is not the case: Imposing a (finite) set of possible trajectories by choosing 
the {Xt}, we have to condition the density matrix at time t on the past measurements. Unlike 
classical probability, even "n on- selective" conditioning influences the state. 

Observe moreover the following fact: 

Proposition VI. 3 The joint probabilities can be re-written in terms of the time reversal 
transitions for the path-conditioned states as: 

4o,n,.....)(^o) = trace (^U^M^^i^,^ ^i^hi^) ■ ■ ^J^^^^^ ■ 

(47) 



Proof. Since [o"t,Xt] = 0, Vt G [0, T], by using (37) we have that: 



{«o,nv,«T) 



>^,o) = trace (n,,(o)4^_,^_^(n,,(i) . . . 7^^^_^ ,^^^_^(n,,(T)a^,Tn,,(T)) . . .)n,,o(o) 
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By noting that wf.^ .^^ .^^(o-o) = wfi^,i,,...,ij,)i^£,o)^ we get the conclusion. □ 

This "backward" representation will play a key role in the solution of the maximum entropy 
problems we discuss in the next Section. 



B. Maximum entropy problems on quantum path spaces 

We consider the simpler maximum entropy problems where only the initial or final density 
matrices are prescribed. The solution to these problems exhibit the same structure of their 
classical analogues, involving a "symmetrized" multiplicative functional transformation. 

Let {S}} be a family of TPCP maps generating a quantum Markov process over [0,T] 
with initial density matrix a^. Assume that at time T the density matrix of the system 
has been found to be pr, being different from the expected cry = o . . . o SQ^ao). Let 
{Xt} be a time-indexed family of observables defining a path space as above. We constraint 
only Xt to be such that [Xt,Pt] = 0, and it admits a spectral decomposition with rank 
one Ilj{T)'s (this is quite natural, since pt is given). Let as above w^(cro) denote the path- 
space distribution induced by the initial condition ctq and the TPCP transitions {S}}. For 
simplicity, in the reminder of the section, the reverse-time quantum operations are assumed 
to be trace preserving. The general case is simply obtained by augmenting the Kraus 



operators in order to have a trace preserving transformation, as detailed in Section |VB 
Consider now the 

Quantum Maximum Entropy Problem (QMEPl): 

minimize {l]){w^{po)\\w^ iao)y,w^ipo) e nipr)} (48) 

with ^(pt) the set of path space distribution induced by a quantum Markov process gener- 
ated by a family of TPCP maps {J-'t} and some initial po such that their path-conditioned, 
final density matrix satisfies pjc- -p = p^. 

In order to solve Problem QMEPl, notice the following facts. 

1. Since we required the nj(T)'s to be rank-one, it follows that for all i G [l,m2-]: 
ni{T)pTU,{T) = trace(pTn,(T))n,(T) = <(po)n,(T). 

Hence, we can write 

W^fo,n,...,*T)(Po) = W^fo,n,...,iT-iNT) • ^^t(Po), (49) 
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where we used the expression (47) and defined the conditional probabilities: 



w 



(io,ii,.--)*T-iKT) 



trace (^,„(o)7^t^ ,,^^(n,,(i) . . . 7^t^^^ ,,^_^(n,,(T)) . . .)n.,(o)) . (so) 



Decomposition (49) clearly plays the role of expressions (??)-( 12) in the classical case. By 
employing (49) and its equivalent for w^j^ iy)('^o)5 we can now write: 



®(wfo,n,...,*T)(Po)lkfio,n,...,*T))(^o)) 



^ ^ ^(io,n,.--,«T) 

«0:«lvi«T 



(Po) log 



w 



(po) 



w 



(io,ii,...,iT)V^oj 



w 



(io,nvi«T-l|«T) 



log 



w 



(iO,nv:«T-l|«T) 
{io,nv,«T-ll«T) 



w^*v(Po) 



+ E-..(^>«)'°s^ 



(51) 



Since /)jf,t = ^j^, ^i^(Po)njj, = pr, notice that the first part of (51 ) is non-negative and inde 



pendent of the initial state, while the second is independent of the choice of f2-^(po) ^ ® (pt)- 
If we can obtain a process with conditioned probabilities such that: 



T f 
111 = 111 

(«0,nv,«T~ll«T) («0,«1v,«T-iI*t)' 



the first part of (51) becomes zero, providing us with an optimal solution. By (50), a 



sufficient condition for optimality is to have the same path- conditioned backward mechanism 
as the reference evolution. We have thus proven the following quantum counterpart of 
Theorem IIII.4I 



Theorem VI. 4 A solution to (QMEPl) (48) is given by the quantum Markov process with 



path- conditioned final density Pt at time T and reverse-time transition mechanism equal to 
that of {St}, namely 

(52) 



ni (■), vtG[o,T-i]. 



Notice that with this optimal choice, the total cost is bounded by the relative entropy of 
the conditioned final density matrices: 



> u>4 Po log / 



Kpt II o'f, 



T ■ 



(53) 



It 
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Let us compute the "forward" quantum operations, which, as in the classical case, will turn 
out to be time dependent even when the reference process is time-homogeneous. By Theorem 
recalling that the conditioned transition mechanism Sj admits a Kraus representation 



V.2 



with operators Hjit + l)Mk(t), see (45), one finds that the Kraus operators of TZ^^^^^ are 
given by the double-indexed Rj^k{£t, (^£,t) = (^jt 

Ij^]^. Reversing this TPCP 

map, now with respect to the state pr,t+i^ we get: 



FjAt) = P>,t+i^;|+i(n,(t + l)Mk{t))al/pP^ 



which can be consider as a non- commutative, "symmetrized" version of a multiplicative 

1 _ 1 

functional transformation in the classical case. In fact, define Nt = Pjr^cy^l- Then we have 
that the TPCP maps of the solution, with initial condition pj^fl., are given by: 

^/(p) = Y.N,^,[li,{t + l)M,{t))Nr'pN^\Mlm,{t + l)) Nl,, 



which has the role of the transitions defined in (16). Moreover, Yt = N^Nt = (^^l Pr,t^st 



satisfies: 



UYt+i) = J]n,(t + i)Mt(t)o-;|_,iP^,,+ia-f_,iM,(t)n,.(t + i) 
-- t -- 

= Z^^£,!RjA£t,a£,t)p:F,t+iRlki£t,a£,t)^£^! =Yt, 

that is, {Yt} is space-time harmonic with respect to the transition St, completing the analogy 
to (16). We remark that, since every time-reversal can be augmented to be TPCP by 
Theorem V.2, one can always complete T^t _ and then J^J{-), to be TPCP. 

Consider now the case where the initial state is constrained to be equal to po, different 
from the a-priori initial condition cq. Consider a path-space induced by observables {Xt} 
such that Xq has non-degenerate spectrum. By using (44), we can write: 

4o,n,...,iT)M = trace (n,,(T)4_,(n,,_,(r - 1)... £t(n.,(0))... )n,,(T)) •4^)(ao) 

= ^i,...,iT|io) ■^o)(^o)- 
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Arguing as above, we get: 



[VJ 



(io,ii,...,iT) 



10 \ll,...,lT («1v.«t)I«o) , 



+ E-.Miog^ 



(54) 



Theorem VI. 5 A solution to (QMEP2) 



minimize {D(w^(/)o)lh^K)); ^^(po) e r](po)} (55) 

with i^(po) the set of path space probability distributions induced by a family of TPCP maps 
{JF/} and initial state po, is given by the quantum Markov process with initial density po and 
forward transitions: 

Tt{-)=St{-), VtG[0,T-l]. (56) 

Remark VI. 6 Altough the QMEP2 problem apparently depends on the choice of the quan- 
tum path-space, that is the observables {Xjj^gp^r]; we remark that its solution does not. The 
difference between problems QMEPl and QMEP2 is given by the fact that in QMEP2 we 
are concerned with the forward transitions, and we do not need to use the path- conditioned 



density matrices (45). The classical case does not present this asymmetry since classical 



non-selective measurements do not alter the state. 

The final cost admits a bound similar to that in Problem QMEPl, that can be easily 
related to the unconditioned states. In fact, using monotonicity of relative entropy with 
respect to conditioning we get: 



E<(/'o)l°g^^fr4 = ^(Po\\^e,o) = Bi£\po)\\£\ao)) < D(p-o||ao), (57) 



^0 



with ^t(p) = ^^n,(o)pn,(o). 

Notice that the operator-sum of the two reverse-time evolutions TZj^^pt^TZe^at satisfy, under 
appropriate restriction on the support of Pi, at: 

1 _i 1 _i 1 _i 

Rk{^t,Pt) = p^Ml{t)pt^\ = p^at ^Rk{Suat)a^^iPt+i, 
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which is again as a quantum symmetrized "multiphcative" functional transformation. To 
carry on the analogy with the classical case, consider two quantum Markov processes cor- 
responding to different initial conditions po 7^ ctq but with same family of trace-preserving 

quantum operations {S}}. Assume for simplicity supp(pt) C supp((Tt), Vt. Define the ob- 

_ 1 _ 1 

servable Yt = ^ptc^ ^. We thus have that: 

-RsAyt) = Y.^;AM,{t)a}af^p,a;K}Ml{t)a;}, = Y,^,. (58) 

k 

This shows that Yt evolves in the forward direction of time with the backward mechanism of 
ati- It is namely quantum space-time harmonic in reverse time with respect to the transition 



of (Tt, once more in striking analogy with (22). Moreover, the special case of property (58) 
where the quantum operation E'^ does not depend on time and cr^ = d" is an invariant density 
matrix for E'^ is, as in the classical case, connected to a strong form of the second law j51j . 



VII. CONCLUSION AND OUTLOOK 



In this paper, we have studied a class of maximum entropy problems on path space 
first for Markov chains and then for quantum channels. In both cases, solutions are ob- 
tained from the "prior" evolution via a multiplicative functional transformation induced by 
a space-time harmonic function in analogy to what was known for the diffusion case. The 
classical theory of Schrodinger bridges is connected to a variety of other fascinating topics 
besides large deviations. First of all, there is Schrodinger's original motivation: He had 
observed the strong analogy between the time reversibility of the solution bridge and that 
of quantum mechanics |i59j . This motivation is from time to time rediscovered in various 
versions of stochastic mechanics see e.g. |35l HQl HHl [56]. Other related topics are recip- 
rocal processes {ID Markov fields), see e.g. EH ES], stochastic control and variational 
principles [HI El], logarithmic transformations of parabolic equations [21], Feynman-Kac 
formula [T6l [29]. etc.. In order to avoid overburdening this first manuscript, we have chosen 
to defer discussing/developing these connections for Markov chains to future publications. 
There is, however, another link: The reverse time space-harmonic functions occurring in the 
solutions of problems (MEP2) and (QMEP2) lead to a strong form of the second law. This 
is presented in 

The Markov chain Schrodinger bridges appear as a flexible tool to be tested on a variety 
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of applications, given the recent surfacing of the full modeling and computational power of 
Markov chains, cf. e.g. [HI [151 12H EH] ■ For quantum systems, this framework may be useful 
to attack steering problems [5] and to complement or improve quantum process tomography 
techinques (see e.g. [39] for a recent review of different methods). Exploring the relations 
of our framework with the theory of quantum error correction [3l [30l EH Hi] appears to 
be a particularly promising research direction. The problem of finding the time-reversal of 
quantum operations or quantum Markov semigroups [H |T0] representing the effect of noisy 
channels on some quantum code is strictly related to many central problems in quantum 
information and its realizations. Moreover, our path-space problems appear to be compatible 
with the general setting proposed in [TT] to develop a quantum version of Sanov's theorem 
for product states. This suggests that our results may play a role in hypothesis testing and 
large deviation theory for quantum Markov evolution, once more in remarkable analogy with 
the classical setting. 
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APPENDIX A: DISCRETE- TIME MARTINGALES 

A general reference is [13]. Following P], we give a somewhat restricted definition of 
martingale. 

Definition A.l Consider a discrete time stochastic process X = {X{t),t > 0} with finite 
or denumerable state space X. The process Y = {Y{t),t > 0} is called a martingale with 
respect to X if 

1. E|r(t)| < oo,Vt > 0; 

2. Y{t) IS a function of {X{s), < s < t}, Vt > 0; 

3. E(F(t + 1)|X(0), X(l), . . . , Xit)) = Yit). 
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We can say that a martingale is conditionally constant. Notice that the case Y{t) = X{t) is 
also included. An elementary example of a martingale is provided by the capital of a player 
at time t in a fair coin tossing game. 

Proposition A. 2 Let h be space-time harmonic for the Markov chain X = {X{t)]t G N} 
with state space X and transition matrix Pit) = {pij{t)). Define the stochastic process 
Y = {Y{t) = h{t,X{t)),t > 0}. Then, if E\Y{t)\ < oo,Wt, Y is a martingale with respect 
to X. 

Proof. We have 

E(F(t + 1) |X(0), X(l), . . . ,X{t)) = E{h{t + 1, Xit + 1)) |X(0), X(l), . . . , X{t)) 

= E{h{t + l,X{t + l))\X{t)). 

Now observe that for alH G A" 

E{h{t + l,X{t + l))\X{t) =i) = + = h{t,i). 

j 

Thus E{h{t + l,X{t + l))\X{t)) = h{t,X{t)) = Y{t). Since properties 1) and 2) are also 
satisfied, we conclude that F is a martingale with respect to X. □ 



APPENDIX B: PROOF OF LEMMA lULl 



Using (??)-( 12), we get 



P n =>>••■ > P ^o, • • • , iT-i, It) log . 



10 «i It 
= E P(^0,^1, • • • ,«T-1,«T 

Observe now that 



.fc=i 



, . . . ,«T-1,«T 
1 



+ log 



IT 



P(zo,«i, . . . ,iT-i,«r)log 

E E P(^0,^l,---,«T-l,«T)log 

It io,ii,--- At-i 

/ ^1 



P: 



It 



TTij,{T) 
1 



P 



It 



P 



It 



T^iTiT) 



{p'MT)) 



27 



Moreover, let p{ik-i,ik) = P(X(fc — 1) = ik-i,X{k) = ik) be the two times marginal of P. 
Observe that p{ik-i,ik) = Qikik-ii^ ~ l)Pifc(^)- We then get 

T 

^ P(io,ii,--.,«r-i,«r) J^lo^ 



k=l 



^ P(«0,^l,---,«T-l,«T)log 

k=l ijT^ikyik-i 

T 

= Y P{^k~uik)^og 



k=l ik,ik 



k=l ik,ik-\ 
T 



likik- 




1) 


^ik'ik- 




1 N 


9ik'ik- 




1) 


^ikik- 


Jk- 


1) 


lik'ik- 


-Ak- 


1) 


^ik'ik- 


Jk- 


1) 


^ikik- 


-Ak- 


1) 


^^k'lk- 


Jk- 


1) 



YY^ (^^^--1 - 1) \Kik-. (k-l)) P.k (k) 



k=l ik 



and (13) follows. 



□ 



APPENDIX C: APPLICATION TO LARGE DEVIATIONS 

The area of large deviations is concerned with the probabilities of very rare events. Let 
Xi,X2,... be i.i.d. random variables with finite expectation. The strong law of large 
numbers asserts that 

1 ^ 

-J^X, ^E(Xi), a.s.. 

1=1 

The study of deviations from the typical behavior is central in probability. We consider first 
normal deviations (called fluctuations by physicists). The central limit theorem implies that 
the quantity 

1 ^ 

P(|-5:.Y.-E(.Y0|>-^) 

i=l ^ 

converges as tends to infinity to a positive value. In (level-1) large deviations one considers 
instead the asymptotic behavior of 

1 ^ 

i=l 

By the law of large numbers, this quantity tends to zero. But one is interested in learning 
if it decays exponentially (it does) and if so with what exponent. We refer to the treatises 
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[T7t UHl [20] for a thorough introduction and to [T5| [55] for large deviations in connections 

to classical Schrodinger bridges. We mention, in passing, that large deviations theory has 

various applications in hypothesis testing, rate distortion theory, etc, see e.g. [13], Chapter 

11], [17, Chapters 2,3,7]. We now investigate the application of the solution to (MEP3) to 

large deviations of the empirical distribution (level-2 large deviations). 

Let A" be a finite or countably infinite set. Let ... be independent, identically 

distributed Markov evolutions on the discrete time interval [0, T] with state space X and 

defined on the probability space P). Let 11 be their common distribution on X^^^. 

We can alternatively think of the Markov chains X* as random variables taking values in 

the cr-compact metric space {X'^~^^,d) where d denotes the discrete metric. The empirical 

distribution fin associated to X^, X^, . . . X" is defined by 

1 " 



\lrSOJ) 



(CI) 



Notice that (CI ) defines a map from to the space Di^X'^^^^ of probability distributions on 



X'^^^ . Hence, if E' is a subset of T>{X^^^\ it makes sense to consider fiuj : fini^^) ^ E). By 
the ergodic theorem, see e.g. [201 Theorem A. 9. 3.], the distributions /i„ converge weakly [HO] 
to n as n tends to infinity. Hence, if II ^ E, we must have F{u! : fin{^) G -E) \ 0. Large 
deviation theory provides us with a much finer result: Such a decay is exponential and the 
exponent may be characterized solving a maximum entropy problem! Indeed, in our setting, 
let E = X>(0,T;p°,p^). Then, Sanov's theorem p] Theorem 8.2], roughly asserts that if the 
"prior" n does not have the required marginals, the probability of observing an empirical 
distribution /i„ in V{0,T;p^,p^) decays according to 



P 



[1^2^^' e V{0,T-p',p')j ~ exp [-ninf {D(P||n);P G V{0,T-p'>,p')}] . 



In the words of [17]: "Sanov's theorem provides a quite unexpected link between Large 
Deviations, Statistical Mechanics, and Information Theory" . Let us go back to the solution 
of problem (MEP3). Since the optimal distribution P of Theorem 



IV. 1 



makes the cost 



function J(P) of Problem 26 equal to zero, we get that the exponent is given by 

D(Piin) 



KP>m - 5^1og<^(0,zo)p° +J2^og^{T,tT)pl 



It 



E 



log 



7Ti^{0)ip{0,io) 



pl + J2^og^{T,tT)pl, 



It 
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where ^ is computed through the Schrodinger system (29)-(30)-(31 ). There are, of course, 
similar large deviations results involving Problems (MEPl) and (MEP2). In the first case, 
where the final marginal is fixed, the exponent is given by D(p^||7r(T)). Symmetrically, in 
the second case, the exponent is D(]9°||7r(0)). 
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